1. Field of the Invention
This invention relates to image processing. More particularly, this invention relates to image segmentation.
2. Description of the Related Art
It is often desirable to obtain a sequence of images in which the background that originally appeared in the images has been replaced with a different background. One popular technique for background replacement is called chroma-key. In this technique, images of the foreground objects or scene are captured in front of a uniformly lit background. This background is usually bright blue (chroma-key is also called xe2x80x98blue-screen imagingxe2x80x99) but may be red or green as well. The technique is completed by replacing those areas of the images that have the background color with corresponding areas of a different background image.
While chroma-key may be used with either photographic or electronic image capture, it is a cumbersome process best performed in a professional studio. Thus, the use of chroma-key for real-time video production and for most consumer applications is limited. Other existing approaches to background replacement are similarly unsuitable for many applications because they require at least two cameras or because they produce inaccurate segmentation results along the foreground-background boundaries.
It is known that a segmentation map may be obtained by comparing an original image having a foreground and background to an image having only the background. However, such a map is likely to contain classification errors caused, for example, by shadows, reflections, and other artifacts within the background that appear only when the foreground objects are added to the scene. It is also known that the number of classification errors may be reduced in some cases by applying an anisotropic diffusion filter to smooth the segmentation map.
A method of applying anisotropic diffusion to image processing was described by Perona and Malik in xe2x80x98Scale-Space and Edge Detection Using Anisotropic Diffusion,xe2x80x99 IEEE Trans. PAMI, vol. 12, no. 7, July 1990, pp. 629-39. This method adjusts each pixel""s value with a diffusion offset that is a function of the pixel""s four nearest-neighbor differences. The function used to calculate the diffusion function must be nonlinear and may require a computationally expensive function such as an exponential. Unfortunately, filters that use anisotropic diffusion are susceptible to impulsive noise and require a large number of iterations to perform a desired degree of smoothing.
It is desirable to obtain a method and apparatus for background replacement that does not require such special devices as a blue screen or more than one camera, requires no prior knowledge about the foreground objects, performs an accurate segmentation of foreground and background, will produce results in real-time, and is rapid and robust to noise.